Skip to content

Core: Add delete_files for removing data files by path - #4459

Open
Anurag-S1ngh wants to merge 2 commits into
apache:mainfrom
Anurag-S1ngh:feat/2787-delete-files
Open

Anurag-S1ngh wants to merge 2 commits into
apache:mainfrom
Anurag-S1ngh:feat/2787-delete-files

Conversation

@Anurag-S1ngh

Copy link
Copy Markdown
Contributor

Rationale for this change

Tables can add existing data files with add_files, but there was no way to remove them without rewriting the table. Java exposes DeleteFiles for this, and PyIceberg had no equivalent.

Add Table.delete_files and Transaction.delete_files, taking the full paths of live data files. Each path must be unique and referenced by the target branch. The call commits a copy-on-write delete snapshot and accepts snapshot properties and a branch, mirroring add_files.

Are these changes tested?

Yes. Unit tests remove files and check the remaining rows and snapshot properties, and cover duplicate paths and paths that are not referenced. tests/table passes.

Are there any user-facing changes?

Yes. A new delete_files API is available on tables and transactions.

Tables can add existing data files but had no way to remove them, which
is the remaining gap with Java DeleteFiles. Add Table.delete_files and
Transaction.delete_files taking full file paths, validating that each
path is unique and still referenced by the target branch, and committing
a copy-on-write delete snapshot with optional properties.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant